Ishaak et al. BMC Public Health 2014, 14:590 

http://www.biomedcentral.com/1471 -2458/1 4/590 BMC 



^BMC 



Public Health 



RESEARCH ARTICLE 



Open Access 



Test implementation of a school-oriented drug 
prevention program "Study without Drugs": 
pre- and post-testing for effectiveness 



Fariel Ishaak , Nanne Karel de Vries and Kees van der Wolf 



Abstract 

Background: In this article, the test implementation of a school-oriented drug prevention program "Study 
without Drugs" is discussed. The aims of this study were to determine the results of the process evaluation and to 
determine whether the proposed school-oriented drug prevention program during a pilot project was effective for 
the participating pupils. 

Methods: Sixty second-grade pupils at a junior high school in Paramaribo, Suriname participated in the test 
implementation. They were divided into two classes. For the process evaluation the students completed a 
structured questionnaire focusing on content and teaching method after every lesson. Lessons were qualified with 
a score from 0-10. The process was also evaluated by the teachers through structured interviews. Attention was 
paid to reach, dose delivered, dose received, fidelity, connection, achieved effects/observed behaviors, areas for 
improvement, and lesson strengths. The effect evaluation was conducted by using the General Liniair Model 
(repeated measure). The research (-design) was a pre-experimental design with pre-and post-test. 

Results: No class or sex differences were detected among the pupils with regard to the assessment of content, 
methodology, and qualification of the lessons. Post-testing showed that participating pupils obtained an increased 
knowledge of drugs, their drug-resisting skills were enhanced, and behavior determinants (attitude, subjective norm, 
self-efficacy, and intention) became more negative towards drugs. 

Conclusions: From the results of the test implementation can be cautiously concluded that the program "Study 
without Drugs" may yield positive results when applied in schools). Thus, this pilot program can be considered a 
step towards the development and implementation of an evidence-based school-oriented program for pupils 
in Suriname. 
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Background 

The test implementation of a school-oriented drug preven- 
tion program "Study without Drugs" was conducted in 
Suriname. Suriname is located in the northeast of South 
America and has an area of 1163,820 km . The population 
consists of ± 600,000 people and is very heterogeneous in 
composition. After a ± 300-year period of colonization, first 
by England and then the Kingdom of the Netherlands, 
Suriname became an independent republic on November 
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25, 1975. On November 25, 1980 a coup d etat took place, 
led by Desi Bouterse. During the period 1980-1987 
Suriname was more often associated with drug trafficking. 
Democracy did his re-appearance in 1987. Studies suggest 
that there are no previously conducted evidence-based 
drug prevention programs developed and implemented in 
schools. "Study without Drugs" is the first school-based 
drug prevention program which during its development 
possible criteria of other effective school-based drug 
prevention programs have been integrated as much as 
possible. This program can serve as a first step towards the 
further development of evidence-based school programs in 
Suriname. 



o 
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School plays an important role with regard to the 
presentation of interventions. Adolescents spend a great 
deal of time in school, and general education and the 
teaching of life skills are central in educational environ- 
ments. Drug prevention programs in schools focus on 
teaching and/or improving skills and knowledge about 
drugs and the behavior determinants of drug use. Teachers 
can help improve and maintain the wellbeing of pupils by 
detecting and preventing drug use [1]. At school, infor- 
mation on drugs can/must be offered because defensible 
pupils perform better [2-5]. Thus, it is important to identify 
effective factors when determining the impact of such pro- 
grams. This is necessary to properly rate the value of the 
executed programs and to make choices for theoretical 
methods and practical strategies. 

In addition to identifying effective factors, school- 
oriented prevention programs must be evidence based. A 
method or approach is evidence based when its function 
has been proved through scientific research [6,7]. 

The evidence should have been found largely through 
'hard' research methods like experiments with randomly 
selected experimental and control groups. Van der Wolf 
and Van Beukering [8] indicated that evidence based 
means that the methods, programs, and approaches used 
have to be scientifically proven to be effective. An evidence- 
based approach promotes reflection on the effectiveness of 
the methods, approaches, and programs used. Influencing 
factors are brought into the picture and analyzed. One 
speaks of evidence-based drug prevention when the strat- 
egies, procedures, and methods used are carefully validated 
and standardized under research conditions and are found 
effective with regard to behavior changes in adolescents 
concerning drug use and drug attitudes [9] . 

A test implementation can be considered small-scale 
evaluation research. Evaluation research is defined as "a 
process for applying academic procedures when gathering 
reliable and valid evidence regarding the manner and the 
scale of which specific activities produce certain results 
and outcomes" [10]. 

Evaluation research measures the effects and outcomes 
of programs and suggests ways for refining these programs 
to obtain more favorable outcomes [11]. 

Process evaluation and effect evaluation 

Process evaluation can focus on various levels (content/ 
organization) and target groups (intermediary and end 
target groups), and can be conducted by applying several 
research methods. Process evaluation is also used to deter- 
mine whether the intervention on each target group was 
implemented uniformly. Process evaluation provides in- 
formation about the reasons behind a program's effective- 
ness or ineffectiveness. Whether or not the intervention 
has an influence is also measured, and the reasons behind 
its success or failure. Many interventions consist of several 



activities. Process evaluations can then be used to deter- 
mine how each activity influences the outcome results 
separately. It is important to know what activities the most 
or the least contributed to this success. This type of 
evaluation will help explaining the relationship between 
specific intervention activities. Process evaluation can help 
to prove why the intervention failed. Examples of failure 
of interventions are: inappropriate intervention design, 
incomplete implementation and insufficient accessibility 
of the target audience, an intervention can be adjusted to 
make it yet successful, through a process analysis. 

Thus, research is conducted on the conditions necessary 
for the effect. Furthermore, insight is given into possible 
improvements or revisions, for example, to improve the 
intervention design and methods for the future [12-14]. 
The following components are also discussed with regard 
to process evaluation: context, reach, dose delivered, dose 
received, fidelity, implementation, and recruitment [15]. 
Not all components need to be included in a process 
evaluation. Priority should be given to questions on the 
size, dose delivered, dose received, and consistency. 

With effect evaluation, the effects on behavior are mea- 
sured. Effect evaluation can focus, for instance, on changes 
in behavior determinants. It is not always possible to meas- 
ure hard, precise, and objective outcomes with health- 
promotion interventions. For this reason, secondary effect 
measures are established and measured. The secondary 
measures are behavior determinants that may have been 
influenced during and because of the intervention. The 
idea is that change in determinants will lead to a change in 
behavior [16]. Thus, two questions can be asked: 1) has the 
program attained its short-term expectations and 2) has 
the program achieved the desired long-term results? 

Drug use among Surinamese adolescents 

The Drug Demand Reduction Program (DDR) conducted 
a nationwide Rapid Situation Assessment in 2005 [17]. 
The assessment studied 480 participants aged 13 years 
and older. The results showed that the majority of smokers 
started smoking before they were 20 years old. 

More than 50% of the youngsters (13-16 year) indicated 
that their fathers smoked. With relation to age, 60% of the 
male respondents had their first alcoholic drink before 
turning 20, as did 38% of the female respondents. 

School surveys conducted in 2004 [18] and 2006 [19] 
by the DDR (by order of CICAD [Inter-American Drug 
Abuse Control Commission]) found alcohol, cigarettes, 
and marijuana to be the most commonly used drugs. 
Cigarettes and marijuana were predominantly used by 
male students aged between 13 and 15 years. Female 
students most commonly used alcohol for the first time 
when aged between 13 and 14 years. Most students who 
smoked were in the second year of high school and those 
who used alcohol were in their first year. On average, 30% 
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of the respondents had access to illicit drugs. For detailed 
data on drug use among Surinamese adolescents, please 
see Tables 1 and 2. 

The DDR (2005) [20] also executed a small-scaled 
study among 125 adolescents aged between 12 and 16. 
The results of this research are as follows: 20% of respon- 
dents stated that they knew someone (or more than one 
person) that used drugs. The drugs users were, for instance, 
direct family members, friends, schoolmates, or strangers 
(e.g., people from the neighborhood and junkies). The most 
commonly used drugs were tobacco (29.6%), and marijuana 
and alcohol (25%). 

Based on the available data, alcohol, tobacco, and 
marijuana are popular among youth in Suriname. Although 
not many young people use cocaine, they could easily point 
out people in their social environment who did. This leads 
to the assumption that cocaine is readily available among 
the Surinamese population. 

Thus, while the level of drug use among youth in 
Suriname is not alarmingly high (yet), prevention measures 
must be implemented to ensure it does not escalate. 

Intervention mapping protocol 

To interpret what has been recorded concerning evidence- 
based school-oriented drug prevention programs, and 
keeping in mind the assumption that the presented data 
from the executed school drug surveys and other (small- 
scale) studies represent just the tip of drug use, a school- 
oriented drug prevention program "Study without Drugs" 

Table 1 Prevalence of drug abuse alcohol and drug abuse 
in secondary schools survey, 2006 



Type of drug Lifetime Last 12 months Last 30 days 





(percentage) 


(percentage) 


(percentage) 


Cigarettes 


35.8 


15.2 


7.9 


Alcohol 


63.5 


46.8 


34.4 


Tranquillizers 


9.8 


5.5 


3.2 


Stimulants 


4.8 


2.7 


1.5 


Solvents & Inhalants 


7.3 


3.4 


2.1 


Marijuana 


6.8 


4.1 


2.3 


Hashish 


1.5 


0 


0 


Hallucinogens 


0.4 


0 


0 


Heroin 


0.5 


0 


0 


Opium 


0.2 


0 


0 


Morphine 


0.3 


0 


0 


Cocaine HCL 


0.6 


0.2 


0.1 


Coca Pasta 


0.7 


0 


0 


Crack 


0.6 


0.3 


0 


Ecstasy 


1.2 


0.2 


0.2 


Other drugs 


3.5 


2 


1 


Any illicit drug 


16.9 


9 


5.2 



Table 2 First use age alcohol and drug abuse in secondary 
schools survey, 2006 



Type of drug Mean 



Cigarettes 


12.9 


Alcohol 


13.2 


Tranquillizers 


13 


Stimulants 


13.7 


Solvents & Inhalants 


11.9 


Marijuana 


15.2 


Hashish 


15.4 


Other drugs 


12.9 


Any illicit drug 


13.7 



was developed using intervention mapping (IM) protocol. 
"Study without Drugs" is a drugs demand-reduction program 
that has been developed for Surinamese adolescents at 
junior high level. The development of the drug prevention 
program was carried out using IM protocol, which con- 
sists of six steps. The added value of this protocol is that it 
lends itself to prepare and execute health-promotion 
activities. IM is based on theory, empirical evidence, and 
additional qualitative and quantitative research, and the 
involvement of stakeholders and actors is central. The six 
phases of the protocol enable health-promotion activities 
to be prepared and executed, taking into account situa- 
tional and personal circumstances. In phase 6, an evalu- 
ation plan is made that focuses on effect and process 
evaluations of the program, and an evaluation model is 
developed [13]. The school-oriented drug prevention 
program developed in March-April 2010 consists of 10 
activities, namely: seven lessons (e.g., Dutch, biology, 
physical education, drawing, religion, and social studies), 
one informative activity, and two evaluation procedures. 
The intervention was carried out from June 2010 until - 
July 2010 (see Table 3). In designing "Study without 
Drugs" attempts have been made to integrate many 
criteria which have been proven to be effective in 
school programs, such as: use of interactive methods; 
"Teaching of Skills" is central; training of persons who are 
going to offer the program is necessary; the research design 
includes a pre-and post-test; an integrated approach and 
position of the prevention program within the curriculum 
is desirable; peers as implementers ensure increased effect- 
iveness; activities that fall outside the curriculum should be 
included; and there is an approach where the family 
(parents), the society or community or parts thereof are 
involved or have an input in the program [21-24]. 

Methods 

Study participants 

The two groups of respondents (all pupils from one junior 
high school) consisted of 60 pupils: 24 boys and 36 girls 
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Table 3 Overview program plan 



Act nr Activity 



Aim 



Means/material 



Conduct pre-test regarding drugs 
(duration 30 minutes) 

Interpersonal information activities 

Duration: 2 hours 

Subject Biology 

Consequences of alcohol, tobacco, 
marijuana and cocaine 

Duration:40 minutes 

Subject Physical Education 



Sports and drugs (steroids) 
Duration: 40 inutes 
Subject Dutch 

Discussion about the consequences 
of swallowing cocaine pellets 

Duration: 80 minutes 

Subject Social Studies 

(Broadcast of play with anti drugs message) 
Duration: 60 minutes 



Subject Dutch 

Write an anti drugs slogan 

Duration: 40 minutes 

Subject Drawing 

Posters with anti drugs message 

Duration: 40 minutes 



Measuring knowledge and attitude with 
regard to drugs 

Ask attention for the drugs issue and 
offer information/knowledge with regard 
to drugs prevention program 

- Categorize mentioned drinks 

- State consequences and have discussion 



■ Educational conversation about ethics 
and mentioning 4 consequences of 
drugs use in sports 

■ Measures to protect against drugs use 



Put events in the right order and discuss them 
ndicate consequences of swallowing pellets 



- Put events in the right order 

- Arrive at a clear view on drugs and youngsters 

- Indicate causes and consequences that 
lead to drugs use among (school going) 
youngsters 

- Think up protective measures 
■ Write and present slogan 



Design poster 



Questionnaire 

Folder, power-point presentation 
drugs in kind 

Developed lesson plan, drugs info 
book, pictures 



Developed lesson plan, pictures, 
for instance of steroids (in kind) 



Developed lesson plan, pictures, 
blackboard, and chalk 



Developed lesson plan, drugs info 
book, pictures of, for instance, 
slogans, blackboard, and chalk 



Developed lesson plan, folders, 
poster, drawing material, chalk, 
drugs info book 



aged between 12 and 15 years, with an average age of 
14 years. Class 2A had 31 pupils (12 boys and 19 girls) 
and 2B 29 pupils (12 boys and 17 girls). It was a select 
sample survey based on voluntary participation. School 
management and teachers committed themselves to carry- 
ing out the test implementation at their school. The group 
of teachers consisted of three women and two men (four 
teachers had been in service for less than 10 years). Two 
teachers were qualified to teach at senior high level, two at 
junior high level, and one had a different teaching qualifi- 
cation. Carrying out the surveys and interviews for both 
the process evaluation and the pre-and post-test were 
done by trained interviewers who were not known to the 
students nor the teaching staff. 

Process evaluation 

The process evaluation by the pupils was done via a ques- 
tionnaire (developed by the researcher) after each lesson. 



The objective of the process evaluation by the pupils was 
to record their opinion on the content of the lessons 
taught and the teaching method. The pupils also got the 
opportunity to express whether or not they enjoyed the 
lessons by qualifying the lesson with a number between 1 
and 10. The pupils had a maximum of 30 minutes to fill 
in the questionnaire. 

A structured interview was also conducted with the 
teachers after each lesson. The interview lasted 45-60 
minutes and consisted of questions on the following 
aspects: reach, dose delivered, dose received, fidelity, 
subject connection, general remarks, lesson strengths, 
and areas for improvement. These interviews were re- 
corded with a voice recorder and later transcribed into a 
computer. 

The transcribed conversations were presented to the 
respondents for possible modification (comments). Then 
the data were coded and the transcripts analyzed. 
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Effect evaluation 

For the effect evaluation, a pre-test was conducted before 
the execution of the program, after two-and-a-half month 
followed by a post-test after the program. Each of the tests 
lasted 35 minutes. Measurements were taken on the basis 
of a one-group pre- and post-test design. These tests 
aimed to measure the pupils' knowledge, skills, and the 
impact of behavior determinants before and after attend- 
ing the prevention program. 

Measuring instruments 

The questionnaire for the effect measurement by pupils 
comprised of 17 propositions aimed to determine the re- 
spondents' opinions on the content (nine propositions) 
and teaching method (eight propositions). 

Measurement was done using a 5-point Likert scale that 
ranged from completely disagree (1) to completely agree 
(5). For the content, aspects such as attractiveness, level of 
difficulty, and use of material were measured. The meas- 
urement of the teaching method included the way in 
which the teacher dealt with the teaching methods, how 
(s)he used the material, and linking to the initial situation. 

Cronbach's alphas for the process evaluation question- 
naires were calculated. The coefficients of the different 
lessons ranged from 0.789 to 0.859. The internal consis- 
tency of coefficients were also checked per subject per 
dimension (content and teaching method). The coefficients 
of the dimension content ranged from 0.659 to 0.808, and 
those for the teaching method from 0.693 to 0.813. 

Table 4 shows the design of the pre- and post-tests. The 
pre- and post-tests have identical content. The Cronbach's 
alphas of both tests were calculated: pre-test, 0.602-0.729; 
post-test, 0.633-0.851. 

Ethical consideration 

Ethical clearance was obtained from the Ministry of 
Education (in particular, the Department of Inspection for 
Continuing Education at Junior level). A letter from the 



Ministry was sent to the participating school in which had 
been indicated that consent for the test deployment was 
granted and cooperation of the management and staff 
were asked for the (test-) implementation. 

With regard to obtaining permission/cooperation of 
the staff, each participating staff member was personally 
approached. Parents of participating students were asked 
by the school for permission to take part. None of the 
parents has objected against their children participating 
in the (test-) implementation. 

Data analysis 

Data were analyzed using Statistical Package for Social 
Sciences (SPSS) software version 19. For the process 
evaluation, the means, standard deviation, ANOVA, 
and Pearson correlation were calculated. For the effect 
evaluation, the pre- and post-test were administered. With 
questions 1, 2, 3, 4, 5, and 7 (see Table 4), two scores were 
possible, namelyl (correct/good) and 0 (incorrect/wrong). 
The total score is the number of correct answers. 

The influence of the behavior determinants was 
measured in question 6. This question is based on the 
Theory of Planned Behavior (TPB). In question 6 pupils 
chose their answer by indicating a score (totally disagree- 
totally disagree) on a 5-point Likert scale. The General 
Repeated Measure (repeated measurement) was applied. 
(By using the General Linear Model (GLM) 'repeated 
measures' a variance analysis was carried out on the inter- 
dependent measurements. This occurs when in the same 
or similar (with the same level of measurement) mea- 
surements were carried out with the same subjects. 
In case N nd measurements are performed on different 
groups, there is - apart from a 'within subjects design' 
also- a 'between subjects design'). 

The advantage of a within- subject design versus a 
between- subject design is that a within- subject design 
is more efficient in the sense that the sample need not 
be less large, while still a significant result is attained. 



Table 4 Description structure for pre- and post-test 



Question Pre-test (Pr-T) 



Question Post-test (Po-T) 



1 Sum up/note 9 types (names) of drugs (Knowledge question [K]) 

2 Indicate which drugs belong to which 14 effects (K) 

3 Recognize 9 drugs from color photographs (K) 

4 Measuring of knowledge about drugs through 18 propositions (K) 

5 Identify 7 aid institutions (K) 

6 Measuring of determinants* of behavior toward drugs through 25 
propositions (Determinant measure) 

7 Measuring of view on control over skills that show defensiveness 
(against drugs) before participating in the program (Skills question) 

* Attitude, subjective norm, observed behavior control, intention. 



1 Sum up/note 9 types (names) of drugs 
(Knowledge question [K]) 

2 Indicate which drugs belong to which 14 effects (K) 

3 Recognize 9 drugs from color photographs (K) 

4 Measuring of knowledge about drugs through 
18 propositions (K) 

5 Identify 7 aid institutions (K) 

6 Measuring of determinants* of behavior toward 
drugs through 25 propositions (Determinant measure) 

7 Measuring of view on control over skills that show 
defensiveness (against drugs) after participating in 
the program (Skills question) 
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This is because the scores of a person are associated 
with each other. This provides additional information, 
making the conclusions more reliable. In a between- 
subject design the averages of the condition will differ 
from each other because in one condition other subjects 
are present rather than in the other condition. In a within- 
subject design, this may cause no difference between the 
averages of the condition, because the same people are 
used for all conditions. The differences between the aver- 
ages of condition parts of a within- subject design are 
therefore less subject to change. Due to the additional 
information of the link, the scores can be compared 
within a person. This makes it possible to assess whether 
the effect occurs consistently in all subjects (there is a 
small MS interaction); that consistency is additional infor- 
mation making the conclusions more reliable. 

We next deal with the results of this measurement, 
and then a chi-square test is used to compare pre- and 
post-measurements. The statistical significance level was 
set at p < 0.05. 

Results 

Qualification of lessons by pupils 

The pupils scored the lessons from 0 to 10. Except for the 
first Dutch lesson, there were no significant differences 
between classes (2A and 2B) and sexes. The average score 
for each lesson was 8.6 or higher, ranging from 8.6 to 9.4. 
A comparison of classes shows that class 2A had a range 
from 8.5-9.7 and class 2B from 8.0-9.3. There is no ap- 
preciable difference based on class and sex. With the boys, 
the social studies lesson scored lowest with 8.0 (range for 
boys: 4-10), while the girls' lowest scoring lessons were 
social studies and biology with 8.7 (range for girls: 6-10). 
A comparison of the two classes shows that the first 
Dutch lesson significantly differed from the other lessons: 
F(2.120)= 0.5.41, p< 0.05. 

Pupils' assessment of lesson content and teaching 
method 

A comparison of the average scores and p-values with 
the assessment of lesson content and teaching method 
did not show any significant differences between sexes 
and classes. The average grade given by pupils showing 
their (positive) assessment was between 38 and 41. 
Based on the SD one can also conclude that male pupils 
(both in the full sample and at a class level) showed a 
greater spread in relation to the average score per lesson; 
however, this difference is not significant. There was 
significant difference between classes 2B and 2A with 
regard to the content of the social studies lesson. 

There was no difference in class and sex with the 
teaching method of the lessons. The average score given 
by the pupils for (positive) appreciation was between 25 
and 32 for the full sample, and in class 2B there was a 



greater dispersion in relation to the average score of the 
lesson. Again, this difference was not significant. 

Relationship between lesson content and teaching method 

The results of the Pearson's correlation show that (p > 0.05) 
there is a strong positive relation between lesson content 
and teaching method (Table 5). 

Process evaluation by teachers 

The text below describes the responses of teachers to 
questions concerning key components: reach, dose deliv- 
ered, dose received, and fidelity. Comments on areas for 
improvement and lesson strengths are also included. 

Reach 

The teachers indicated that they checked attendance at 
every lesson (7 lessons for each class). All 31 pupils from 
class 2A attended 6 lessons. Only in one lesson three 
students were absent. 

All 29 pupils from class 2B attended 6 lessons. Where 
in one session two pupils were absent. 

Thus, over 80% of pupils per class participated in the 
drugs prevention program. According to registration de- 
tails, a total of 60 pupils participated in both the pre- 
and post-test. 

Dose delivered 

The biology teacher indicated that the one of the lesson 
objectives, to teach the "consequences of tobacco, alcohol 
and marijuana use", was not achieved due to a lack of 
time. The rest of the teachers stated that the objectives 
were reached. 

The social studies teacher found that the teaching 
method of 'asking questions' after showing a DVD did not 
work due to a lack of time. Furthermore, he remarked that 
not all of the lesson phases were dealt with in enough 
depth. For organizational reasons, he reduced the intro- 
duction because time had been lost during preparation. 

Regarding the drawing lesson (making an anti-drugs 
poster), the teacher found that the pupils got quite 
stressed trying to finish their poster within the allowed 
time. The teacher also had to repeat the instructions and 
criteria a number of times to ensure the students knew 
what they were doing. 

When showing the Spanish-language film with English 
subtitles, the Dutch teacher had to explain some sections 
as a small number of pupils asked for clarification. The 
teacher commented that it is advisable to work with 
Dutch subtitles, or to provide a short explanation or 
summary of the film before, during, or immediately it 
is shown. 
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Table 5 Pearson's correlation between lesson content and teaching method: full sample (N = 60) 

Biology content PE content Dutch 1 content Drawing content Dutch 2 content Social studies content Dutch 3 content 

Biology 0.846** 
Method 

PE 0.887** 
Method 

Dutch 1 0.871** 



Method 

Drawing 0.768*" 

Method 

Dutch 2 

Method 

Social studies 

Method 

Dutch 3 

Method 



**p>0.01. 

Dose received 

The teachers reported being satisfied with the availability 
and use of the material. They estimated that most of the 
material was suitable for the level and age of the pupils, 
and that it was ready for use. The teachers also noted 
that the majority of the lessons based on a didactical 
model (i.e., didactical analysis). The person-to-person 
instruction round preceding the program was deemed 
sufficiently supportive. The teachers indicated that the 
trainer/instructor gave the trainees enough room to put 
forward their own views on lesson content and teaching 
methods. Biology and Dutch teachers thought parts of 
the text of the drugs information book was difficult for 
the pupils to understand. The physical education teacher 
said that the pictures helped to clearly show the effects 
of steroid use. 

The physical education teacher also stated that both 
the information book and folders were used well by the 
pupils when executing their assignment (making an anti- 
drugs poster). With the Dutch 3 lesson, the film was 
judged to be a realistic portrayal of the consequences of 
drugs smuggling. 

There was very little deviation from the instructions 
on how to use the material. All teachers indicated that 
they would use the material again. 

Fidelity 

All lessons were executed in accordance with the lesson 
plans provided. The lesson phases were dealt with in the 
indicated order. Prescribed material, instructions, and 
teaching methods in the lesson plans were largely imple- 
mented in accordance with the instructions. 



0.560** 

0.761** 

0.792' 



Areas for improvement and lesson strengths 

The teachers also noted areas for improvement and lesson 
strengths. These comments are shown in (see Table 6). 

Effect evaluation results 
Listing known drugs 

In the pre- and post-test pupils were asked to list all the 
drugs they knew of. The variable time was found to have a 
significant main effect pre-test: F(l. 56) = 5.68, p = 0.000; 
post-test F(1.56) = 8.81, p = 0.000. The average shows that 
the number of known drugs was higher after the interven- 
tion than before it. 

Although there is no significant main effect, class 2A 
showed a greater knowledge of drugs than class 2B. Girls 
showed more progress than boys (this difference is not 
significant, see Table 7). 

The chi-square test indicates significant differences 
between the full sample with the pre-test and the full 
sample with the post-test, because a sizable increase was 
noticeable with all drugs when they were mentioned. 
The 10 drugs mentioned were cocaine, tobacco, alcohol, 
steroid, hashish, xtc, marijuana, crystal meth, and opium. 
With all 10 drugs, an increase can be noted with the num- 
ber of drugs mentioned in the post-test (see Table 8). 

Coupling of drugs with effects of use 

Fourteen possible drug effects from drug use were 
indicated. Pupils were asked to write down one drug 
associated with each of the 14 effects. The variables time 
F(l, 56) = 42.94, p = 0.000 and class F(l, 56) = 5.91, p < 0.05 
were found be significant main effects in the pre- and 
post-test. Furthermore, it should be noted that there is a 
significant interaction effect between time and sex and 
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Table 6 Lesson strengths and areas for improvement 



Strengths 


Areas for improvement 


Biology 1 


Biology 1 


Pictures are good; clearly show effects of alcohol and tobacco use and that pupils must 


Putting up pictures takes time; many minutes get lost 


distinguish between alcoholic and non-alcoholic drinks in the introduction 


that way — perhaps a slide show 


Physical education 


Physical education 


When dealing with several forms of drugs and narcotics, you could see that the pupils 


Have a short talk beforehand to prepare the pupils for the 


were shocked by the effects of drugs on the body 


images 


Dutch 1 


Dutch 1 


Presentation of the slogans by the pupils; this way they could express their creativity 


Working together could be better 


Drawing 


Drawing 


Every pupil was motivated 


Executing the assignment takes time and there was too 




little time 


Good collaboration 


Not every member of the 5-member group was active 


They were satisfied with the result 




Dutch 2 


Dutch 2 


Good focus on competition; incites them to take action to win 


A prize at the end would be nice for the pupils 


Thanks to the game the pupils remember the information better and can apply it 




Social studies 


Social studies 


We II -executed play in class 2A 


Not all pupils watched the film with their full attention 


Good participation when thinking up causes and effects 




Dutch 3 


Dutch 3 


Realistic portrayal of the swallowing of drugs and drug smuggling 


Dutch subtitles preferable to English subtitles and 


Clear information presented about effects 


translation without subtitles would be better 



time and class. The average score was higher in the post- 
test (see Table 7). 

The results from the effect and drug coupling exercise 
showed a significant difference for nine effects when 
comparing the pre- and post-test results for the full 
sample. Five of these effects showed a positive shift in 
the post-test. Observed differences concern the following 
items: baldness and hair loss (Chi 2 = 26.12; df = 1; p < 0.05), 
red eyes (Chi 2 = 6.42; df= 1; p < 0.05), male hair growth 
(Chi 2 = 14.90; df = 1; p < 0.05), damage to mucous mem- 
branes (Chi 2 = 8.29; df=l; p< 0.05), development of 
anxiety (Chi 2 = 15.09; df=l;p< 0.05 0), coma (Chi 2 = 8.16; 
df= 1; p < 0.05), headache (Chi 2 = 8.64; df= 1; p < 0.05), 
cirrhosis of the liver (Chi 2 = 33.41; df = 1; p < 0.05), and 
switching to hard drugs (Chi 2 = 7.35; df = 1; p < 0.05). 
Four effects showed a decrease in the post-test. Three 
showed an increase in the post-verification, but were 
not significant. Two items/effects showed a significant 
decrease in the post-test. None of the pupils successfully 
coupled the effect dizziness in the pre-test. 

Recognizing drugs in photographs 

The tests also included 10 colored photos clearly showing 
various drugs. The pupils were asked to identify the drugs 
and write the drug name beneath the photos. 

Both time F(l, 56) = 105.69, p = 0.000 and class F(l, 56) = 
16.93, p < 0.05 were found to be main effects. A noticeable 



difference (increase) can be seen in the averages between 
the measurements taken before and after the test. The sig- 
nificant interaction effect between time and class shows 
that class 2A was slightly better at recognizing drugs 
in the post-test than 2B (see Table 7). Taking into ac- 
count the recognition of each drug separately, a comparison 
of the full sample pre-test and post-test shows that there is 
a significant difference in time because of an increase in the 
recognition of nine of ten drugs depicted (see Table 9). 

Propositions on drugs 

Eighteen propositions were presented with the intention 
to measure knowledge about drugs and drug-related 
items. There were significant main effects with the vari- 
ables time F(l, 56) = 53.33, p = 0.000, sex F(l, 56) = 53.31, 
p < 0.05, and class F(l, 56) = 68.61, p = 0.000. The post-test 
averages were higher than the pre-test. There was 
also a significant interaction effect between sex and class 
F{1, 56) = 13.22,/? = 0.000 and between time, sex and class 
FX, 56) = 11.76, p = 0.000 (see Table 7). 

Regarding the impact of time, classes 2A and 2B showed 
a positive shift in the post-test. Male and female pupils in 
class 2A scored better in the measurement beforehand 
than those in class 2B (see Tables 7 and 10). 

When comparing the full sample in the pre- and post- 
tests, the groups differ significandy with regard to 7 of the 
18 propositions, namely: proposition 5, marijuana is also 



Table 7 Mean (and SD) for time, sex, and class per question in the pre-test and post-test 





Time 


SD Time 


Time 


SD 


Male 


SD Male 


Male 


SD Male 


Female 


SD Female 


Female 


SD Female 


2A pre 


SD 2A 


2A 


SD 2A 


2B 


SD 2B 


2B 


SD 2B 




Pre- 
test 


Pre- 
test 


Post- 
test 


Post- 
test 


Pre- 
test 


pre-test 


Post- 
test 


Post- 
test 


Pre- 
test 


Pre- 
test 


Post- 
test 


Post- 
test 


Test 


Pre- 
test 


Post- 
test 


Post- 
test 


Pre- 
test 


Pre- 
test 


Post- 
test 


Post- 
test 


Listing known 
drugs 


5.68*** 


3.7 


8.81*** 


3.9 


5.75 


3.51 


8.95 


0.204 


5.63 


3.87 


8.72 


0.454 


4.8 


4.24 


8.87 


0.34 


6.62 


2.79 


8.75 


0.435 


Coupling drugs 
with effects of use 


1 .63*** 


1.14 


2.95*** 


i .33 


1.41* 


0.717 


3.41* 


1.47 


1.77* 


1.35 


2.63* 


0.882 


1.61* 


1.34 


2.48* 


1.15 


1.65* 


1.39 


3.44* 


1.35 


Recognizing drugs 
in photographs 


2.73*** 


1.75 


5.35*** 


1.76 


2.95 


1.82 


5.04 


1.98 


2.58 


1.71 


5.55 


1.59 


1 .32*** 


0.54 


5.67*** 


1.68 


4.24*** 


1.77 


5.00*** 


1.81 


Propositions 
on drugs 


7.88*** 


2.29 


1 1 .95*** 


3.67 


7.33* 


2.76 


10.41* 


3.91 


8.25* 


1.87 


12.40* 


3.17 


8.83 


1.67 


11.87 


3.15 


6.86 


2.44 


11.34 


4.13 


Identifying aid 
organizations 


1 .38*** 


0.884 


3.58*** 


1.78 


1.58 


1.13 


3.7 


1.98 


1.25 


0.64 


3.5 


1.63 


1.67 


1.1 


3.74 


1.78 


1.06 


0.37 


3.41 


1.7 


Skills for 
defensibleness 


6.15 


2.47 


6.61 


3.78 


5.33 


2.64 


6.45 


3.62 


6.69 


2.22 


6.72 


3.93 


6.87*** 


2.66 


9.90*** 


1.64 


5.37*** 


2.02 


3.10*** 


1.58 


Attitude 


42.96* 


8.49 


46.64* 


7.32 


4.46 


7.11 


49.5 


4.6 


42 


9.14 


45.03 


8.11 


41.26 


7.84 


44.78 


4.49 


44.4 


8.9 


48.22 


8.85 


Intention 


11.25*** 


5.25 


19.25*** 


5.94 


11,95 


5.57 


19.63 


5.46 


10.83 


0.507 


19,02 


6.28 


10.58** 


5.21 


1 7.03** 


6.88 


12.03** 


5.28 


21.81** 


3.17 


Behavior control 


17.35* 


3.88 


21.94* 


4.02 


18.60* 


3.1 


21.39* 


5.56 


16.55* 


4.15 


21.39* 


5.56 


22.30* 


2.65 


21.16* 


5.05 


16.96* 


3.23 


22.75* 


2.4 


Subjective norm 


18.65*** 


6.54 


22.79*** 


1.96 


19.13 


4.96 


23 


1.63 


18.36 


7.39 


22.66 


2.07 


20,61 


4.91 


22.96 


1.49 


16.4 


7.5 


22.59 


2.4 



*p > 0.05; **p > 0.01 ; ***p = 0.00. 
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Table 8 Full sample results for the pre-and post-tests: listing known drugs (N = 60) 



Drugs 


TOTGR Pre-test % 


TOTGR Post-test % 


p-value 


Drugs 


TOTGR Pre-test % 


TOTGR Post-test % 


p-value 


Cocaine 


83.0 


100 


p < 0.05 


Hashish 


63.3 


100.0 


p < 0.05 


Tobacco 


61.0 


100 


p < 0.05 


Opium 


60.0 


81.7 


p < 0.05 


Alcohol 


65.0 


100 


p < 0.05 


XTC 


65.0 


100.0 


p < 0.05 


Steroids 


66.7 


100 


p < 0.05 


Crystal meth 


65.0 


100.0 


p < 0.05 



seen as a gateway drug (Chi 2 = 3.75; df= 1; p < 0.05); 
proposition 9, the lungs are harmed by tar in tobacco 
smoke {Chi 2 = 6.00; df= 1; p < 0.05); proposition 10, drugs 
can change, stimulate and numb one's consciousness 
(Chi 2 = 4.68; df= 1; p < 0.05); proposition 11, crystal meth 
affects the entire body (Chi 2 = 8.78; df = 1; p < 0.05); prop- 
osition 15, marijuana comes from the plant Cannabis 
sativa (Chi 2 = 5.91; df= l;p< 0.05); proposition 17, doping 
is used to achieve top performances in a dishonest manner 
(Chi 2 = 4.48; df= 1; p < 0.05); and proposition 18, pure co- 
caine looks like fine crystalline powder (Chi 2 = 7.06; df= 1; 
p < 0.05). 

Identifying aid organizations 

In both the pre- and post-tests, pupils were presented 
with an overview of 18 institutes; students had to iden- 
tify 7 as aid agencies. For time, the main effect is signifi- 
cant with both measurements taken before and after the 
test (F(l, 56) = 92.68, p = 0.000). 

Pupils were better able to indicate aid organizations in 
the post-test. There appears to be a significant combined 
interaction effect between time, sex, and class (F(l, 56) = 
4.35, p = 0.000). Regarding the impact of time, the boys in 
class 2B experienced a greater positive shift in the 
post-test. Furthermore, the girls in class 2A always scored 
better in the pre- and post-tests than those in class 2B 
(see Tables 7 and 10). 

An analysis of the results for pupils' recognition of aid 
organizations shows that the pupils differ significantly 
with all 10 instances and in the post-test (see Table 11). 

Behavior determinants 

Time is a significant main effect for attitude F(l, 56) = 6.24, 
p = 0.000. The post-test showed that pupils now have a 
more negative attitude towards drugs. Furthermore, sex is 
also a main effect for attitude F(l, 56) = 4.11, p = 0.000. 



The attitude of both girls and boys showed a more positive 
shift in the post-test. 

Girls have a much more negative attitude towards 
drugs in the post-program test. With regard to the sub- 
jective norm, time F(l, 56) = 20.63, p = 0.000 and class 
F(\, 56) = 5.30, p = 0.000 are found to be mains effects. 
The averages of both classes (2A and 2B) were higher in 
the post-test. This indicates that pupils became more sen- 
sitive towards their own appraisal of the social pressure to 
use drugs or not. 

Looking now at the behavior determinant, observed be- 
havior control, the variables time F(l, 56) = 39.5, p = 0.000 
and sex F(l, 56) = 0.487, p = 0.000 are shown to be main 
effects. The averages were higher in the post-test. The 
variables sex and class have a significant interaction effect 
F(l, 56) = 5.05, p = 0.000. Although the boys showed a 
positive shift in the post-test, the girls showed a greater 
improvement. Thus, the girls were better able to control 
their behavior regarding drugs (use). With regard to the 
effect of time on class, 2A had a better initial score, but 
2B showed greater improvement. 

With intention there was a main effect for time F(l, 56) = 
66.31, p = 0.000 and class F(l, 56) = 8.45, p = 0.000. There 
was an increase in the averages, which indicates that the 
pupils experienced a positive shift: there was an increase 
in the intention to show/execute behavior against drugs. 
Class 2B had a higher average than class 2A in both the 
pre- and post-tests (see Table 7). 

Skills for defensibleness 

Twelve propositions were presented representing a 
skill that indicates defensibleness against drugs. The variable 
class has a main effect. Based on class F(l, 56) = 118.08, 
p < 0.05), a difference (increase) can be observed be- 
tween the averages of the measurements before and 
after. Class 2A showed a greater positive shift in the 
post-test. 



Table 9 Full sample results for pre-test and post-test: identification of drugs (N = 60) 



Recognized drugs 


TOTGR Pre test % 


TOTGR Post test % 


p-value 


Drugs 


TOTGR Pre test % 


TOTGR Post test % 


p-value 


Steroids 


100.0 


83.3 


p < 0.05 


XTC 


8.3 


80.0 


p < 0.05 


Crack 


3.3 


58.3 


p < 0.05 


Crystal meth 


3.3 


13.3 


p < 0.05 


Hashish 


1.7 


48.3 


p < 0.05 


Marijuana 


70.0 


91.0 


p < 0.05 


Opium/poppy 


23.3 


70 


p<0.5 


Cocaine 


20.0 


71.7 


p < 0.05 


Marijuana 


10.0 


16.7 


p > 0.05 


XTC 


6.7 


60.0 


p < 0.05 



Table 1 0 Mean (and SD) for propositions on drugs and aid organizations in the pre-test and post-tests, with a significant interaction effect of time, sex, and class 





Male 


SD male 


Male 


SD male 


Male 


SD male 


Male 


SD male 


Female 


SD female 


Female 


SD female 


Female 


SD female 


Female 


SD female 




2A 


2A 


2A 


2A 


2B 


2B 


2B 


2B 


2A 


2A 


2A 


2A 


2B 


2B 


2B 


2B 




Pre-test 


Pre-test 


Post-test 


Post-test 


Pre-test 


Pre-test 


Post-test 


Post-test 


Pre-test 


Pre-test 


Post-test 


Post-test 


Pre-test 


Pre-test 


Post-test 


Post-test 


Propositions 


8.66** 


1.85 


1 3.08** 


2.35 


6.00** 


2.95 


7.75** 


3.30 


8.94** 


1.61 


12.15** 


3.57 


7.47** 


1.87 


1 3.88** 


2.44 


on drugs 


































Identifying aid 


2.16* 


1.40 


3.58* 


1.88 


1.00* 


0 


3.83* 


2.16 


1.36* 


0.76 


3.84* 


1.77 


1.11* 


0.48 


3.11* 


1.40 


organizations 



































*p>0.05; **p>0.01. 
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Table 1 1 Full sample results for the pre-test and post-test: identification of aid organizations (N = 60) 



Institutions 


Pre-test 


Post-test 


p-value 


Institutions 


Pre-test 


Post-test 


p-value 


Tabernacle of faith 


13.3 


55.0 


p < 0.05 


NieuweGrond 


8.3 


28.3 


p < 0.05 


PCS 


67.0 


71.0 


p < 0.05 


Victory out reach 


6.7 


41.7 


p < 0.05 


Kick the habit 


11.7 


51.7 


p < 0.05 


De Stem 


13.3 


35.0 


p < 0.05 


Nieuwe grand 


8.3 


28.3 


p < 0.05 


BAD 


3.3 


60.0 


p < 0.05 



Two skills showed a significant decrease in the post-test, 
one remained stable and nine skills showed a positive shift 
(increase) (see Table 7). 

Discussion 

Process evaluation 

The interviews with the teachers showed that they con- 
sidered the process evaluation of the drug prevention 
program to accurately represent the success of the pro- 
gram. The teachers' positive reactions regarding the imple- 
mentation and/or adoption of the interventions are based 
on the following characteristics: appropriate level of 
complexity, can be communicated, can be included with 
existing curricula, observability of activities to be executed 
and results, possibility to adapt content through rever- 
sibility and modification, reformulating instructions and 
getting feedback [25-27]. 

Personal characteristics like attitude, subjective norm, 
observed behavior control, and the intention of the re- 
cipient or person delivering the intervention contribute 
to the experiences/opinions [28]. The factors that impact 
on the willingness of teachers to pass on the content of 
an intervention are as follows: the subjective norm, own 
effectiveness, interpretation of one's job, the extent in 
which one feels responsible for passing on the content, 
and instrumentality of materials [29,30]. That is, the ex- 
tent in which the material has clear instructions for its 
preparation, presentation, and evaluation and the extent 
in which the material leads pupils to be active and enthu- 
siastic [31-33]. 

In the process evaluation, it was mentioned that in some 
instances, time constraints led to improvisation. With 
other health-promotion programs, times limitations have 
been described as bottlenecks [34-37]. 

With regard to dose delivered, it can be concluded 
that the teachers worked fairly well with the (sub) goals 
of the lessons and that those goals were achieved. With 
regard to dose received, the teachers reported being 
satisfied with the intervention. The material was received 
positively; it was largely suitable for the age and level of 
the pupils. With regard to its reach, the majority of pupils 
participated in the program. 

The assessment/ qualification of lessons by pupils depends 
on the way in which they (participants in an innovation/ 
intervention) experience the session. Factors like a 
pupil's initial level, attractiveness of the lesson content 



and materials, complexity of the lesson, instructional 
skills and expertise of the teacher, and application/use 
of multimedia appear to affect the performance of pupils 
[26,31,33,38,39]. Group dynamics processes can also affect 
a class's assessment/qualification, as well as assessment by 
sex. Group dynamics can affect an individual's perform- 
ance, making it either better or worse [40,41]. 

Effect evaluation 

The differences that became apparent through the chi- 
square test can be explained with the taxonomy of Bloom 
[42] and Anderson et al. [43]. Educational psychology 
indicates that pupils can score considerably better with 
less complex cognitive activities/goals. These cognitive 
activities can include listing and/or recognizing drugs 
(reproduction of knowledge). 

A comparison of the results of the pre- and post-tests 
showed an increase in the percentage of correct re- 
sponses with the listing of drugs, recognition of drugs, 
and aid organizations. There is, however, no great in- 
crease in knowledge when coupling effects with the cor- 
rect drugs, propositions on drugs, and mastery of skills 
[44,45]. It is difficult to give a precise reason for the oc- 
casional decreases in the post-test. It is possible that an 
overestimation by the pupils occurred in the pre-test, 
resulting in a more flattering picture of the situation. 
The post-test is possibly a more realistic representation 
because the pupils were confronted with accurate infor- 
mation at the intervention. With replication research, 
the execution of tasks can also be measured by means of 
observation. 

With the measurement of the effect of time on sex and 
class, class influence is a feature of the group dynamics 
processes [40,46]. Items that are of influence in the class 
are, among others, the number of pupils, composition of 
the group, group atmosphere and the resulting social 
interaction, and the group objectives [33]. 

With regard to the impact of the pupils' sex, such char- 
acteristics can also affect the overall school performance. 
There are clear differences between the performances of 
female and male pupils. 

Females profit more form individual learning than 
from collaborative learning. Males profit from both to 
the same extent. Divergent patterns in the collaboration 
between females and males show a cognitive chasm 
between female and male pupils. There are also likely 
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to be differences in the way they elaborate, mode of 
communication, and their leadership styles [47-49]. 

According to Veendrick, Tavecchio, and Doornenbal 
[50] the learning performance of boys can also be affected 
by the phenomenon whereby education is becoming femi- 
nized. Thus, boys go without male role models, with all its 
consequences. The difference in learning and performance 
motivation between individuals in groups/classes may play 
an important part. Thorpe [51] stated that boys are 
expected not to like school, as reinforced by comments 
from their parents and teachers. This has negative conse- 
quences for learning motivation and performance. Girls 
experience just the opposite and they do like school, thus 
performing better. A third explanation in the literature is 
a difference in learning styles. A learning style is the 
manner in which somebody best acquires knowledge. One 
person learns better, for instance, when the material is 
presented visually, while another learns best when the 
content is presented aurally [52]. Boys and girls presum- 
ably differ with regard to learning styles, and presumably 
those used in schools are not appropriate for boys. Van 
Langen and Driessen [53] have stated that boys perform 
well in mathematics and physics. They prefer learning 
rules and abstract facts by heart, and respond to episodic, 
factual, and detailed comments. Girls prefer open-ended 
tasks that are coupled with real situations. They respond 
more extensively to questions, putting lessons learned in a 
broad context. 

Conclusion 

The dual purpose of "Study without Drugs" test-imple- 
mentation was to determine the results of the process 
evaluation and whether the Proposed school-oriented drug 
prevention program was effective for the participating 
pupils when offered as a pilot. 

From the results of the evaluation process can be 
stated that: 

all classes were assessed as to be sufficient by the 
students. Furthermore, the students involved were 
positive about the content of the courses offered and 
the teaching methods used. Teachers had for the large 
part a quite positive opinion on the following aspects: 
reach, dose delivered, dose received, fidelity and. 
subject connection, 

With regard to the measurement of the effect, it can be 
noted with caution that: participating pupils showed a 
(slight) increase in knowledge of drugs (that is, in the field 
of listing known drugs, recognizing drugs in photographs, 
identifying aid organizations) and their drug-resisting 
skills. There was no significant increase in knowledge 
regarding coupling of drugs with effects of use, Proposi- 
tions on drugs). Furthermore, after attending the program, 



a number of behavior determinants — attitude, subjective 
norm, self-efficacy and intention — were more negative 
towards drugs. 

Some limitations of this test implementation were: 
The test implementation has been done on a small scale, 
therefore it is desirable to note that the results cannot 
be generalized to other groups of students outside the 
group of respondents (no external generalizability). 

The results (both process and impact) may become 
affected by aspects such as: impact test, instrumentation 
and maturation, previous experience concerning drug 
prevention programs offered outside the intervention. 

The design of the study subject is pre-experimental, 
with no control group. Respondents were selected with- 
out randomization. Necessary adaptations must be made 
namely: application of randomization since this reduces 
the risk of selection bias. The research design should be 
in line with the experiment and control group in which 
respondents who are randomly selected are also to be in- 
cluded in both. Matching of the research -i populations 
should take place at the start of the study design. Further- 
more, it is possible when analyzing data from a study con- 
ducted to perform "stratify or statistical control" (through 
multivariate regression analysis -i) afterwards. Previously 
enumerated improvements may lead to increase of inter- 
nal and external validity and generalizability of results. 

Finally, it can be said that the developed program "Study 
without Drugs" - after adjustments on the basis of the re- 
corded points for improvement/limitations observed - can 
be used as a starter for further development of evidence- 
based school drug prevention programs. A structural im- 
plementation of the program into the school curriculum 
could lead to (improvement/ increase) enhance the quality 
of the lives of adolescents as well as reducing the risk that 
they will use drugs. 
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